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ABSTRACT 

This article employs confirmatory factor analysis to evaluate the factorial validity and the 
cross-national comparability of the UTAUT constructs with respect to mobile learning in higher 
education in four Caribbean countries. Except for the measurement of one factor, the UTAUT 
constructs exhibit adequate reliability and validity. Though full metric invariance is not achieved, 
cross-national comparisons of the regression relationships among the factors are still possible. In 
addition, non-invariant item intercepts also affect the comparisons of the factor means. Partial 
scalar invariance is required. 

Keywords: UTAUT, measurement invariance, Caribbean, mobile learning, technology adoption, 
higher education 


INTRODUCTION 

The increasing use of technology in higher education leads to the increasing importance of 
educational technology acceptance. This is relevant to the Caribbean region where the use of 
e-learning systems is accompanied by numerous challenges (Waldron 2009). It is important to 
identify the variables that influence user acceptance as this can help in ensuring successful 
delivery of education. In this regard, the Unified Theory of Acceptance and Use of Technology 
(UTAUT) (Venkatesh etal. 2003) identifies some important factors. The UTAUT model is based on 
the Theory of Reasoned Action (TRA) (Fishbein & Ajzen 1975), and it incorporates components of 
several other models inclusive of the Technology Acceptance Model (TAM) (Davis 1989) and the 
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modified Technology Acceptance Model (TAM2) (Venkatesh & Davis 2000). The UTAUT model 
has become popular in technology acceptance studies, but such studies focus overwhelmingly on 
Western (Schepers & Wetzels 2007; Traxler 2007) and Asian countries. The appropriateness of 
the measurements in these contexts does not guaranteed their validity in the Caribbean region. 
Furthermore, the Caribbean may themselves differ in their experience with mobile technology and 
such differences can affect measurement comparability (Li & Kishore 2006). Both the validity and 
cross-national comparability of the measurements therefore need to be demonstrated rather than 
assumed. 

This paper investigates the validity and comparability of the UTAUT constructs across four 
Caribbean territories. The constructs are evaluated in the context of mobile learning in higher 
education. Many definitions of mobile learning are currently in use. However, mobile learning is 
essentially learning with the aid of mobile technologies. This can occur at anytime and anywhere 
via mobile devices (El-Hussein & Cronje 2010). This ubiquitous element of mobile learning sets it 
apart from e-learning in general. Mobile devices are distinct from more traditional technologies 
such as a computer which requires either a fixed position (desktop) from which access to the 
internet can be obtained or which facilitates access only at hotspots or other specific areas 
(laptop) (Jeng et al. 2010). Mobile devices include for example mobile phone, tablets and others 
which facilitate internet access from anywhere and therefore facilitates more flexibility (El-Hussein 
& Cronje 2010; Hlodan 2010). In addition to providing new evidence which can guide the use of 
the UTAUT model in mobile learning adoption in the region, this paper provides results that are 
relevant to the study of technology adoption in general. They aid determination of the 
generalizability of the UTAUT measurements outside of the frequently studied contexts and add to 
the evidence about the cross-national comparability of the measures. 

THE UTAUT MODEL 

The UTAUT factors are Performance Expectancy (PE), Effort Expectancy (EE), Social Factors 
(SF), Facilitating Conditions (FC), Behavioural Intention (Bl) and Use Behaviour (UB) (Venkatesh 
et al. 2003). PE is the extent to which the individuals believe that the technologies improve their 
performance. EE is the perceived ease of use. SF is the degree to which the respondents believe 
that significant persons in their lives think that they should use the technologies. FC is the 
respondents' beliefs about the extent to which organisational and technical infrastructure to 
support the use of the technologies exist. Bl is the behavioural intention to use the technologies. 
UB measures the intensity of use. Given the engagement in mobile learning in the Caribbean is 
voluntary, measuring UB in relation to mobile learning is difficult. As such, UB is not evaluated in 
this paper. 

The items included in the UTAUT instrument are usually adapted for the specific research domain; 
for example, acceptance of information systems, virtual learning environment, mobile learning in 
higher education acceptance and use of IT (Al-Gahtani et al. 2007; Bandyopadhyay & Fraccastoro 
2007; Cheon et al. 2012; Jairak et al. 2009; Nassuora 2012; Teo 2011; Van Raaij & Schepers 
2008; Wang & Shih 2009; Tibenderana et al. 2010; Attuquayefio & Addo 2014). Modifications of 
the items are made in this study to ensure that the content is applicable to mobile learning (see 
Table 1). 
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Table 1: Measurement of the UTAUT Constructs 


Construct Code Item 


PE 

PEI 


PE2 


PE3 


PE4 

EE 

EE1 


EE2 


EE3 

SF 

SF1 


SF2 


SF3 

FC 

FC1 


FC2 


FC3 


FC4 

Bl 

BI1 


BI2 


BI3 


Mobile Technologies are useful in education in general. 

Using mobile technologies enable students to accomplish tasks more quickly. 
Mobile technologies would improve students' performance. 

Mobile technologies would increase students’ productivity. 

Mobile technologies are easy to use. 

Finding or using features in mobile technologies is easy. 

Learning to operate mobile technologies is easy. 

People who influence my behaviour think that I should use mobile technologies. 
People who are important to me think that I should use mobile technologies for 
learning. 

University teachers are supportive of the use of mobile technologies. 

In general, my University campus has support for mobile learning. 

I have the resources necessary to use m-Learning. 

I have the knowledge necessary to use m-Learning. 

Support from an individual or service is available when problems are 
encountered with m-Learning technologies. 

I intend to use m-Learning technologies in the next semester. 

I predict I will use m-Learning technologies in my courses in the next semester. 

I have a plan to use m-Learning technologies in the near future. _ 


• Scale labels: 1 - Strongly disagree , 2 - Disagree, 3 - Neither Agree nor Disagree, 4 - 
Agree, 5 - Strongly Agree. 

• In the items, m-Learning refers to mobile learning. 

• Mobile learning, mobile technologies and the other terms used in the items were defined 
for the participants at the beginning of the survey and the definitions were repeated at 
intervals throughout the questionnaire. M-Learning means mobile learning. 


In the UTAUT model, PE, EE and SF along with their interactions (in some cases) with age, 
gender, experience and voluntariness of use explain Bl, while Bl and SF explain UB (see Figure 1) 
(Venkatesh et al. 2003). Venkatesh et al. (2003) indicates that the model explains approximately 
70% of the variance in Bl. However, such a high proportion of explained variance is not usually 
found in other studies. Some studies have reported explained variances as low as 35% to 45% 
(Thomas et al. 2013; Teo 2011), but others have reported larger explained variances in the range 
50% to 65% (Tibenderana et al. 2010; Al-Gahtani et al. 2007). Nevertheless, the validity and 
reliability of the UTAUT measurements and the utility of the model in explaining Bl are widely 
acknowledged. 

The literature on the UTAUT model includes investigations of measurement comparability. In 
some cases, the cross-national/cross-cultural comparability is considered (Im et al. 2011; Kang et 
al. 2011; Oshlyansky et al. 2007) whereas in others, measurement comparability between other 
groups is investigated (Li & Kishore 2006). Oshlyansky et al. (2007) show that the UTAUT items 
measure the same factors in several Western and non-Western countries; however, the level of 
comparability demonstrated (configural invariance: discussed subsequently) is not enough to 
permit direct comparisons of regression relationships or factor means. Kang et al. (2011) and Im et 
al. (2011) investigate the comparability of the UTAUT measures in greater detail between the 
United States and Korea and find instances of a lack of measurement comparability leading to 
biased regression relationships. Li and Kishore (2006) focus on groups defined in various ways; 
for example, gender and experience and they also find instances of measurement incomparability. 
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Figure 1: Conceptual UTAUT Model 


Cross-national comparisons are often regarded as cross-cultural comparisons especially when 
countries which are known to differ culturally are included. This can be said of the comparisons 
between the United Stated and Korea (Im et al. 2011). Culture is known to affect measurements 
differentially and it is therefore not surprising that there are violations of comparability between 
such groups (Van de Vijver & Poortinga 1997). Even when the groups are not defined by culture, 
comparability of the UTAUT measures is not guaranteed (Li & Kishore 2006). This underscores 
the need for demonstrations of the absence of such measurement violations before substantive 
measures such as regression relationships and factor means are compared. 

In this paper, the validity and cross-national comparability of the UTAUT constructs are evaluated 
with respect to mobile learning in the Caribbean region. In the region, mobile learning has not yet 
been formally adopted by universities. As such, the use of mobile technologies for education is a 
voluntary undertaking by both the students and the teachers. In spite of this, there is evidence to 
suggest that mobile learning is in use. Research on e-learning, technology and mobile learning 
adoption in the Caribbean region is ongoing and researchers have documented such evidence in 
various territories; for example, in Barbados (Gay et al. 2006), Guyana (Gaffar et al. 2011; 
Thomas et al. 2013), Jamaica (Reid & Levi 2008) and Trinidad and Tobago (Kalloo & Mohan 2011; 
Sultan & Mohan 2009). In addition, Figaro-Henry et al. (2011) note that facilitators and students 
are willing to embrace the use of mobile technologies for learning in the region. However, we know 
of only one study that provides an evaluation of the UTAUT factors. This evaluation is done with 
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data from Guyana and the measurements appear to be valid (Thomas et al. 2013). However, this 
is one territory and in spite of the fact that the UTAUT model is based on several technology 
adoption theories and that it has been evaluated in many countries, it is still important to establish 
the validity of the measurements before the model is applied in a new context. Given the almost 
absence of evaluations of the model, the Caribbean region qualifies as a new context. 
Furthermore, given the evidence of violations of measurement comparability elsewhere, an 
evaluation of comparability in the Caribbean region where studies on technology adoption are 
emerging, is also an important contribution. 


CROSS-NATIONAL COMPARISONS 

Cross-national and group comparisons in general, assume the absence of bias and hence, the 
preservation of the psychometric properties or scales across the groups (Chen 2008; Meredith 
1993; Schmitt & Kuljanin 2008; Vandenberg & Lance 2000; Van de Vijver & Poortinga 1997). 
There are three main types of bias - construct, method and item bias - that affect group 
comparisons (Van de Vijver & Leung 1997). This article focuses mainly on construct and item bias 
which result from the survey items used (Van de Vijver & Tanzer 2004) and which affects 
measurement comparability. 

Construct bias means that the constructs measured by the items are not the same in all the 
groups. This is caused by several issues among which are differential appropriateness of the 
content of the items and incomplete overlap in the definitions of the constructs across the groups 
(Van de Vijver & Leung 1997). Item bias refers to measurement artefacts at the item level (Van de 
Vijver & Leung 1997). Item bias alos has several sources and they include incidental differences in 
the content of the items, poor translation and poor item formulation (Van de Vijver & Tanzer 2004). 
Construct and item bias may occur even when the same items are used with the same wordings 
due to differences in the frame of reference of the groups. These two types of bias can be detected 
to a large extent by evaluations of measurement invariance (Ml). 


MEASUREMENT INVARIANCE 

Ml implies independence of the observed item scores from group membership (Meredith 1993; 
Millsap 1995). Hence, with Ml achieved, individuals with the same true standing on a construct 
have the same observed scores (Schmitt & Kuljanin 2008). If this does not hold, group and 
cross-national (group) comparisons are invalidated (Byrne & Watkins 2003; Van de Vijver & 
Tanzer 2004). The demonstration of Ml is therefore necessitous to cross-national (and group) 
comparisons (Cheung & Rensvold 1999). 

Ml tests form a hierarchy in which the lower levels are less restrictive and are prerequisite to the 
higher levels. The first recommended Ml test is an omnibus test of equality of the between-group 
variance-covariance matrix (Joreskog 1971). If this test lacks significance, the data from the 
different groups may immediately be pooled since there will be no group differences (Joreskog 
1971; Schmitt & Kuljanin 2008; Vandenberg & Lance 2000). However, most researchers begin 
with the test for configural invariance instead of the very stringent omnibus test. In fact, the three 
levels of Ml that are most often useful in group comparisons are configural, metric and scalar 
invariance (Van de Vijver & Leung 1997). These three level of Ml are investigated in this paper. 

Configural invariance is the lowest level of Ml and it focuses on the basic form of the model. It 
asserts that an equal number of factors is formed in each group (Horn & McCardle 1992; Joreskog 
1971) and that there is a fixed pattern of salient and non-salient factor loadings (Steenkamp & 
Baumgartner 1998). Although it is required for the other levels of Ml, configural invariance is 
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insufficient for group comparisons of factor means, regression and other structural relationships 
which are often the focus of research. Configural invariance is affected by construct bias (Van de 
Vijver & Leung 1997). 

The next level Ml is metric invariance which indicates that the measurement units (or interpretation 
of the items) are preserved across the groups. This is evaluated by imposing between-group 
equality constraints on the respective factor loadings (Dimitrov 2010). Metric invariance permits 
group comparisons of the structural relationships (factor variances, covariances and regression 
effects) among the factors, but not of the factor intercepts (means) (Dimitrov 2010). Metric 
invariance is affected by method and construct bias (Van de Vijver & Leung 1997). Comparisons 
of factor intercepts require scalar invariance which in turn requires between-group equality of the 
item intercepts in addition to metric invariance (Schmitt & Kuljanin 2008; Vandenberg & Lance 
2000). Scalar invariance indicates that the basic item levels are equal across the groups and it is 
affected by construct, method and item bias (Van de Vijver & Leung 1997). 

The described Ml procedure implies full invariance at each step, but this does not always happen 
in practice. One or more items may show non-invariance and the restrictions on such items may 
be relaxed leading to partial invariance. For partial invariance, at least two items must be invariant 
-the reference indicator and one other item (Cheung & Rensvold 1999). Under partial invariance, 
the freed items do not contribute to the group comparisons (Byrne et al. 1989). When the bias is 
severe and when several items are biased, partial invariance can result in substantial changes in 
the meaning of the construct (Millsap & Kwok 2004). As such, it should be applied with caution and 
the modifications to the measurement of the constructs should be taken into consideration when 
the results are interpreted. 


THE CARIBBEAN CONTEXT 

The Caribbean region consists of developing countries that differ with respect to several variables 
including human development and tertiary level education (See Table 2, for information about the 
four territories under study.). The most recent United Nations Human Development Report 
identifies Latin America and the Caribbean as the most unequal region in human development 
globally (United Nations Development Programme 2013). For example, whereas Barbados, 
Jamaica, and Trinidad and Tobago have achieved the CARICOM’s target 15% participation rate 
for tertiary education that was set in 2002, most other countries, including Guyana, have not 
(Tewarie 2009). The ICT rankings of the territories are also markedly different (Table 1) and there 
are large variations in their ICT development index 1 . Barbados and Trinidad and Tobago tend to 
outperform Jamaica and Guyana with Barbados being among the highest ranked countries in the 
world whereas Guyana is ranked quite low (International Telecommunication Union 2013). There 
are also large between-territory differences in the percentage of individuals that use the Internet. 
Apart from the need for evaluations of the UTAUT measures due to the relative novelty of the 
model in the Caribbean, the heterogeneity of the territories may affect the measurements. 

Differences in tertiary education levels can indicate differences in ICT adoption and development 
(Lee 2001). Combined with the disparities in the ICT development and rankings, these realities 
support the view that the local conditions may create differences in experience with technology 


1 The ITU, ICT development index is a summary measure of 10 indicators which evaluate ICT 
access, usage and skills within a country. The scale of the index ranges from 1 to 10 and the 
values may be used to compare countries. The ITU, ICT rankings give the rank position 
compared to the other countries based on the development index (International 
Telecommunication Union 2013). 
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across the Caribbean territories. Differences in experience can in turn lead to a lack of 
comparability of the UTAUT constructs. In particular, experience is shown to limit the comparability 
of the UTAUT effort expectancy and facilitating conditions measures (Li & Kishore 2006). 
Experience with mobile technology can affect the frame of reference of the population leading to 
both construct and item bias. As a consequence, the meaning of some of the items and their 
average levels can vary even though the same items are administered with the same wording. We 
do not anticipate, that construct bias will play a major role, but we expect that item bias affects the 
measurements. As such, it is more likely that violations of scalar invariance will occur than 
violations of metric invariance. 


Table 2: Country Variables 


Country 

Population 

Size# 

Gross 

Enrolment 

Ratio 

(Tertiary, 

2011)+ 

UNDP 
Human 
Development 
Index Rank+ 

Percentage 

of 

Individuals 
Using the 
Internet 

ICT 

Development 

Index* 

ICT 

Development 

Rank* 

Barbados 

283,221 

65.90 

38 

73.33 

6.65 

36 

Guyana 

795,369 

11.90 

118 

34.31 

3.08 

105 

Jamaica 

2,712,100 

29.00 

85 

46.50 

3.68 

93 

Trinidad and 
Tobago 

1,337,439 

11.50 

67 

59.52 

4.73 

66 


*ICT information is for the year 2013 and is obtained from ITU (International Telecommunication 
Union 2013). A Obtained from the ITU Indicators Database 2012 (International Telecommunication 
Union 2012). + Human Development Report 2013, UNDP (United Nations Development 
Programme 2013). # The World Bank (The World Bank 2013). 


Our expectations in relation to measurement invariance, is based on the belief that the UTAUT 
factors will be measured adequately by the items. The UTAUT model has been evaluated widely 
and even though, a lack of measurement invariance is found in some cases, the model appears to 
be quite robust. The evidence from Guyana also indicates that with some limitations, especially in 
relation to the FC, the factors are adequately measured (Thomas et al. 2013). However, these 
results do not guarantee the validity of the measurements in the Caribbean region in general. As 
such, validity also needs to be demonstrated. 


DATA AND METHODS 
Data 

This study is done with data collected from students at several university campuses within the 
Caribbean region. The data were collected via a web survey (See Table 1 for the UTAUT items) 
administered in: Barbados at the UWI Cave Hill, Guyana at the University of Guyana, Jamaica at 
the University of Technology and UWI Mona, Trinidad and Tobago at the UWI St. Augustine and at 
the UWI Open Campus, between October 2012 and February 2013. The students were contacted 
by email and invited to participate on a voluntary basis without any incentives. The email contacts 
were made through the university which ensured that the entire university student population was 
contacted in each case. In total, 1726 respondents completed the questionnaires: 649 (Barbados), 
243 (Guyana), 262 (Jamaica: 112 (University of Technology), 150 (UWI Mona), 333 (Trinidad and 
Tobago), and 239 (UWI Open Campus). Because the Open Campus pulls students primarily from 
several territories within the Caribbean region, it is excluded from the analysis. Consequently, the 
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effective sample consists of 1487 students. These groups are regarded as coming from the 
various university-territory combinations, hence, the data from the campuses in Jamaica are 
merged. 


Table 3: Sample Distributions 


Gender 



Territory 



Barbados 

Guyana 

Jamaica 

Trinidad & Tobago 

Male 

175 (0.72) 

151 (0.62) 

181 (0.75) 

189 (0.78) 

Female 

68 (0.28) 

92 (0.38) 

62 (0.26) 

54 (0.22) 


The approximate sample proportions are enclosed in brackets. 


The large differences in sample sizes, would result in large power differences in the measurement 
models across the groups. Power will be comparatively high especially for Barbados. To address 
this, the sample sizes are scaled to that of the smallest group (Guyana) by making simple random 
selections from the data. The gender distributions of the initial data are preserved in the selections 
(see Table 3). However, we note that these distributions are not necessarily representative of the 
population distributions. For example, we know generally that there are more females than males 
at the University of Guyana, but the sample consists of more responses from males. In the 
absence of the population distributions, we are unable to apply weights (for example through 
iterative proportional fitting) to adjust for nonresponse due to self-selection. This is a limitation of 
this study. In spite of this, the results are still expected to be indicative of what can be expected of 
the measurements within the various country-campus contexts since the same self-selection 
issues affect each sample. 

Given the focus on Ml, it is necessary to discuss the inclusion of UWI campuses in three of the 
four territories (Barbados, Trinidad and Tobago, and part of Jamaica). Although three of the 
campuses come under the UWI brand, they exist in different territories which are at different 
stages in their ICT and economic development (see Table 2). The country conditions such as 
physical resources and experience with and access to ICT resources are expected to influenced 
the conditions at the campuses. This is expected to create sufficiently different local conditions 
despite the common university name. However, that the possibility of influences of the UWI brand 
is not eliminated is a limitation of the current study since this can result in measurement 
invariance. This is particularly relevant to Barbados and Trinidad and Tobago where students from 
only UWI campuses are included in the samples. 


Methods 

Several methods for evaluating Ml are available, but Multi-Group Confirmatory Factor Analysis is 
both the most powerful and the most popular method (Meuleman et al. 2009). In the analysis, the 
respondents' ratings are regarded as continuous and the models are computed with robust 
maximum likelihood estimation using Mplus 7.11. In the models, the covariances among the 
UTAUT factors are freed (see Figure 2). 

Configural invariance is judged from the basic form of the measurement models when they are 
estimated separately in each group. Only the factor loadings and the residual variances implied by 
the UTAUT constructs are initially estimated (see Figure 2). If modifications are required, they are 
discussed. Once the adequately fitting models are established in each group, they separate 
models are combined and estimated simultaneously to provide the baseline, configural invariance 
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model. For metric invariance, the respective factor loadings are equated with each other across 
the groups. Modifications to the equality constraints on the factor loadings are allowed if 
warranted, but this is done in a stepwise manner. To evaluate scalar invariance, equality 
constraints are imposed on the respective item intercepts across the groups in addition to the 
equality constraints on the factor loadings. Only the intercepts of items whose loadings are 
invariant are included in the evaluation of scalar invariance. 



Figure 2: The UTAUT Measurement Model 


The simultaneous model estimations are done in three stages. In the first stage, all the two-group 
combinations are evaluated. This results in the evaluation of six models consisting of two groups 
each. In the second stage, the three-group combinations are evaluated; four models in total. 
Finally, in the third stage, a four-group model consisting of all the territories (Barbados, Guyana, 
Jamaica and Trinidad & Tobago) is evaluated. 

Before Ml is assessed, the reliability and factorial validity of the UTAUT constructs in each group 
are evaluated. Cronbach's alpha greater than or equal to 0.70 is taken as indicative of adequate 
reliability (Flair et al. 2006). The results of the model estimations also enable evaluation of the 
practical significance of the factor loadings; standardised loadings greater than or equal to 0.70 
are regarded as ideal. Factor convergent validity is achieved if the average variance extracted 
(AVE) is greater than or equal to 0.50, and discriminant validity is achieved if the square root of the 
AVE is greater that the factor correlations (Fornell & Larcker 1981). These are discussed for the 
model in each territory separately before joint estimation. 

The fit of the models is evaluated based on alternative fit indices since the chi-square statistic is 
too sensitive for large sample sizes. In particular, the Root Mean Square Error of Approximation 
(RMSEA) less than or equal to 0.06, Comparative Fit Index (CFI) greater than or equal to 0.95, 
and the Standardised Root Mean Square Residual (SRMR) less than or equal to 0.05 are 
regarded as indicative of adequate global fit (Byrne 2012; Flu & Bentler 1999). To determine the 
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level of Ml achieved, the relative fit of the nested models are judged based on the change (A ) in 
the fit indices. In particular, ARMSEA > 0.0 land A CFI > 0.005 indicate significantly poorer fit 
for metric, scalar and strict invariance whereas A SRMR > 0.025 indicates lack of metric 
invariance and A SRMR > 0.005 indicates poorer fit for scalar (sample sizes less than 300) (Chen 
2007). These criteria are used in combination and the decisions are based on a majority of the 
indices (Sass 2011). In spite of a strong research tradition of using global fit indices to evaluate 
factor models, misspecifications may still occur when these indices indicate adequate fit (Saris et 
al. 2009; Van der Veld 2008). Such misspecifications can be detected with the use of the program 
Jrule for Mplus 0.91 (Oberski 2008). Jrule (judgment rule) for Mplus, is a program that takes the 
Mplus output as its input and it uses a combination of the expected parameter change, 
modification index and power (all obtained or calculated automatically from the Mplus output) to 
detect parameter misspecifications (Saris et al. 2009; Van der Veld 2008). In this study, high 
power is set at 0.80 and Type I error at 0.05. The misspecification is set to 0.10 for error 
covariances and at 0.40 for factor loadings. 


RESULTS 
Construct Validity 

For each group, the initial model with the five UTAUT factors fits adequately with respect to the 
RMSEA and the CFI. The fit is a bit poorer with respect to the SRMR but not poor enough to cause 
great concerns (see Table 4). The models are therefore accepted as fitting adequately in general, 
however, reliability and validity of the factors are examined furtherqw. 


Table 4: Within-Country Models 


Country/ Group 

X* 

Degrees of freedom 

RMSEA 

CFI 

SRMR 

Barbados 

211.14 

109 

0.06 

0.95 

0.06 

Guyana 

220.14 

109 

0.06 

0.95 

0.06 

Jamaica 

200.57 

109 

0.05 

0.97 

0.06 

Trinidad & Tobago 

207.66 

109 

0.05 

0.96 

0.06 


Except for SF and FC in Guyana, the reliability (Cronbach’s alpha) of each of the scales for each 
construct in each group is above 0.70 (Table 4). The reliabilities of SF and FC in Guyana (0.65 and 
0.66 respectively) are only marginally lower. Flence, the reliability of each scale in each group is 
adequate. 

Turning attention to the size of the factor loadings (Table 5), we observe that many of them exceed 
0.70, but that some are lower. These loadings in combination with the factor level convergent 
validity (AVE: Table 5) and the discriminant validity (Table 6) lead to a few remarks about the 
factors. Firstly, the respective indicators are all valid measures of PE, EE and Bl and both the 
convergent and discriminant validity of these factors are confirmed since the average variance 
extracted (AVE) exceed 0.50 and their correlations with the other factors are lower than the square 
root of the average variance extracted. Secondly, the third indicator of SF (SF3) has limited validity 
in each group; especially Barbados and Guyana where the loadings fall below 0.40. In spite of this, 
both the convergent and discriminant validities of the factor (SF) are adequate owing largely to the 
very high validity of the two remaining items. Thirdly, the validity of the first and fourth indicators of 
FC (FC1 and FC4) are relatively low except for FC4 in Jamaica. However, both items still appear 
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to be useful measures of the construct since in each case, the factor loading exceeds 0.40. This 
factor also shows limited convergent validity overall especially in Guyana (AVE = 0.34), but it 
provides unique information since it also shows adequate discriminant validity. 


Table 5: Item Loadings 


Item Barbados Guyana Jamaica Trinidad & Tobago 



P 

E 


F 


P 

E 


F 


P 

E 


F 


P 

E 


F 



E 

E 

SF 

C 

Bl 

E 

E 

SF 

C 

Bl 

E 

E 

SF 

C 

Bl 

E 

E 

SF 

C 

Bl 


0 . 





0 . 





0 . 





0 . 





PEI 

68 





66 





68 





72 






0 . 





0 . 





0 . 





0 . 





PE2 

78 





68 





75 





75 






0 . 





0 . 





0 . 





0 . 





PE3 

79 





86 





87 





86 






0 . 





0 . 





0 . 





0 . 





PE4 

79 

0 . 




88 

0 . 




86 

0 . 




79 

0 . 




EE1 


86 





83 





85 





84 






0 . 





0 . 





0 . 





0 . 




EE2 


86 





89 





91 





90 






0 . 





0 . 





0 . 





0 . 




EE3 


82 

0 . 




81 

0 . 




86 

0 . 




78 

0 . 



SF1 



86 





80 





91 





86 






0 . 





0 . 





0 . 





0 . 



SF2 



92 





88 





88 





97 






0 . 





0 . 





0 . 





0 . 



SF3 



36 

0 . 




28 

0 . 




51 

0 . 




44 

0 . 


FC1 




60 





51 





53 





57 






0 . 





0 . 





0 . 





0 . 


FC2 




80 





70 





75 





77 






0 . 





0 . 





0 . 





0 . 


FC3 




70 





56 





68 





75 






0 . 





0 . 





0 . 





0 . 


FC4 




57 

0 . 




55 

0 . 




70 

0 . 




47 

0 . 

BI1 





95 





92 





97 





94 






0 . 





0 . 





0 . 





0 . 

BI2 





92 





81 





90 





92 






0 . 





0 . 





0 . 





0 . 

BI3 





85 





79 





74 





84 


0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

AVE 

58 

72 

57 

45 

82 

60 

71 

50 

34 

71 

63 

76 

62 

45 

77 

61 

71 

62 

43 

81 


0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

0 . 

Alpha 

84 

88 

73 

77 

93 

85 

88 

65 

66 

87 

87 

90 

80 

76 

90 

86 

88 

78 

74 

92 


Factor loadings are the fully standardised loadings of the models in separate estimations. AVE - 
average variance extracted. Alpha - Cronbach Alpha. 
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Table 6: Discriminant Validity 


Group 

Construct 

PE 

EE 

SF 

FC 

Bl 

Barbados 

PE 

0.76 






EE 

0.07 

0.85 





SF 

0.07 

0.08 

0.75 




FC 

0.07 

0.09 

0.09 

0.67 



Bl 

0.05 

0.06 

0.07 

0.06 

0.91 

Guyana 

PE 

0.77 






EE 

0.07 

0.84 





SF 

0.06 

0.07 

0.71 




FC 

0.09 

0.10 

0.08 

0.58 



Bl 

0.06 

0.07 

0.08 

0.08 

0.84 

Jamaica 

PE 

0.79 






EE 

0.08 

0.87 





SF 

0.07 

0.07 

0.79 




FC 

0.07 

0.07 

0.08 

0.67 



Bl 

0.07 

0.06 

0.06 

0.07 

0.88 

Trinidad & Tobago 

PE 

0.78 






EE 

0.07 

0.84 





SF 

0.07 

0.08 

0.79 




FC 

0.07 

0.09 

0.09 

0.66 



Bl 

0.05 

0.06 

0.07 

0.06 

0.90 


The square root of the AVE is placed on the diagonal and is highlighted. The off-diagonal elements 
are the correlations between the respective factors. 


Comparability of the UTAUT Measures 

By accepting the model for each group, we have in essence verified configural invariance. The 
basic form of the model is the same in each group and the models confirm to the specification of 
the UTAUT theory. In the next stage, factor models contain two groups each are evaluated. The 
configural invariance models are obtained by simultaneous estimation of the pairs under 
consideration. 

The results for the two-group models (see Table 7) indicate that full metric and scalar invariance 
are achieved for the comparisons of Jamaica and Trinidad & Tobago, Jamaica and Barbados, 
Barbados and Trinidad & Tobago. In the evaluation of scalar invariance between Barbados and 
Trinidad & Tobago one item intercept (PEI) has a high modification index. At the same time, the 
power of the Jrule test for this path is larger than the 80% threshold while the expected parameter 
change (EPC) is not very large. The test therefore appears to be too sensitive in this case, and 
hence the large modification index for the parameter is ignored. 

Whenever, Guyana is included in a two-group evaluation, violations of Ml are encountered. Full 
metric invariance of the measurements is observed when Guyana is compared to Jamaica and 
Trinidad & Tobago, but when compared to Barbados the loading of the second indicator of EE 
(EE2: Finding or using features in mobile technologies is easy.) is higher in Guyana indicating that 
the validity of this item as a measure of EE is lower in Barbados than Guyana. Nevertheless, given 
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that the loading is generally large and that the expected change in the parameter is not (0.17), the 
bias resulting from comparisons based on this non-invariant item is not expected to be large. 


Table 7: Measurement Invariance Evaluation 

Model v 2 df A y 2 Adi RMSi ARMS CFI A CR SR A SRA Mo EP Po Pat 

A A MR d. C we h 


Ind r 


Two-Group Models 

Groups: Jamaica and Trinidad & Tobago 


Configural 

408. 

21* 

2 

1 

8 



0.0 

50 


0.9 

62 


0.05 

7 


Metric 

427. 

53* 

2 

3 

0 

19. 

32 

12 

0.0 

50 

0.000 

0.9 

61 

-0.0 

01 

0.06 

3 

0.00 

6 

Scalar 

456. 

72* 

2 

4 

2 

29. 

18* 

12 

0.0 

51 

0.001 

0.9 

56 

-0.0 

05 

0.06 

4 

0.00 

1 

Groups: Jamaica and Barbados 








Configural 

411. 

70* 

2 

1 

8 



0.0 

55 


0.9 

59 


0.06 

0 


Metric 

431. 

2 

q 

19. 

12 

0.0 

-0.00 

0.9 

-0.0 

0.06 

0.00 

13* 

o 

0 

43 

52 

3 

58 

01 

7 

7 

Scalar 

473. 

94* 

2 

4 

2 

42. 

80* 

12 

0.0 

55 

0.003 

0.9 

50 

-0.0 

08 

0.07 

2 

0.00 

5 

Groups: Jamaica and Guyana 








Configural 

420. 

65* 

2 

1 

8 



0.0 

52 


0.9 

56 


0.06 

0 


Metric 

437. 

2 

q 

17. 

12 

0.0 

-0.00 

0.9 

0.0 

0.06 

0.00 

95* 

o 

0 

30 

51 

1 

56 

00 

6 

6 

Scalar 

510. 

12* 

2 

4 

2 

72. 

16* 

12 

0.0 

58 

0.007 

0.9 

39 

-0.0 

17+ 

0.07 

4 

0.00 

8 

Pscalarl 

454. 

50* 

2 

4 

1 

16. 

54* 

11 

0.0 

51 

0.000 

0.9 

54 

-0.0 

02 

0.06 

8 

0.00 

2 

Groups: Guyana and Barbados 








Configural 

431. 

26* 

2 

1 

8 



0.0 

55 


0.9 

49 


0.06 

2 


Metric 

458. 

41* 

2 

3 

0 

27. 

15 

12 

0.0 

55 

0.000 

0.9 

46 

-0.0 

03 

0.07 

2 

0.01 

0 


-0. 0.3 FC1 
42 6 :GY 


-0. 0.5 EE2 
17 6 :BB 
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Model 


df 

A dj RMSi ARMS CFI 

A CR 

SR 

MR 

A SRA Mo 

d. 

EP 

C 

Po 

we 

Pat 

h 












Ind 


r 


Pmetricl 

443. 

2 

9 

12. 

11 

0.0 

-0.00 

0.9 

0.0 

0.06 

0.00 





93* 

9 

67 

54 

1 

50 

01 

7 

5 





Pscalarl 

586. 

2 

A 

142 

11 

0.0 

0.015 

0.9 

-0.0 

0.08 

0.01 

63. 

0.3 

0.6 

FC1 

00* 

4 

0 

.07* 

69 

+ 

13 

37+ 

2 

5+ 

10 

5 

2 

:BB 

Pscalar2 

508. 

2 

q 

64. 

10 

0.0 

0.006 

0.9 

-0.0 

0.07 

0.00 

31. 

-0. 

0.5 

FC3 

60* 

o 

9 

67* 

60 

35 

15+ 

5 

8 

00 

26 

7 

:BB 

Pscalar3 

472. 

2 

q 

28. 

Q 

0.0 

0.002 

0.9 

-0.0 

0.06 

0.00 

11. 

-0. 

0.8 

BI3: 

09* 

O 

8 

16* 


56 

45 

05 

9 

2 

88 

12 

4 

BB 

Groups: Guyana and Trinidad & Tobago 










Configural 

427. 

77* 

2 

1 

8 



0.0 

54 


0.9 

52 


0.06 

0 






Metric 

446. 

91* 

2 

3 

0 

19. 

14 

12 

0.0 

53 

-0.00 

1 

0.9 

52 

0.0 

00 

0.06 

6 

0.00 

6 





Scalar 

530. 

2 

A 

83. 

12 

0.0 

0.008 

0.9 

-0.0 

0.07 

0.00 

37. 

0.3 

0.4 

FC1 

03* 

4 

2 

12* 

61 

32 

20+ 

5 

9 

11 

1 

9 

:TT 

PScalarl 

485. 

2 

A 

38. 

11 

0.0 

0.002 

0.9 

-0.0 

0.07 

0.00 

10. 

-0. 

0.4 

FC3 

09* 

4 

1 

18* 

55 

44 

08 

0 

4 

49 

17 

7 

:TT 

Pscalar2 

472. 

2 

/I 

25. 

10 

0.0 

0.001 

0.9 

-0.0 

0.06 

0.00 

10. 

-0. 

0.7 

BI3: 

56* 

4 

0 

66* 

54 

47 

05 

8 

2 

98 

12 

8 

TT 

Groups: Barbados and Trinidad & Tobago 











418. 

2 

1 

8 



0.0 


0.9 


0.05 






Configural 

82 



54 


56 


9 







430. 

2 

q 

11. 

12 

0.0 

-0.00 

0.9 

0.0 

0.06 

0.00 





Metric 

23 

o 

0 

41 

52 

2 

57 

01 

4 

5 






462. 

2 

/I 

32. 

12 

0.0 

0.002 

0.9 

-0.0 

0.06 

0.00 

12. 

-0. 

0.8 

PEI 

Scalar 

23 

4 

2 

00* 

54 

52 

05 

7 

3 

54 

12 

5 

:TT 






Three-Group 1 

Models 








Groups: Guyana, Barbados 

and Trinidad & Tobago 









Configural 

638. 

91* 

3 

2 

7 



0.0 

54 


0.9 

53 


0.06 

0 






Metric 

677. 

3 

C 

38. 

24 

0.0 

-0.00 

0.9 

-0.0 

0.06 

0.00 

11. 

-0. 

0.4 

EE2 

01* 

O 

i 

11 

53 

1 

52 

01 

9 

9 

67 

20 

1 

:BB 

Pmetric 

661. 

95* 

3 

4 

23. 

05 

22 

0.0 

52 

-0.00 

2 

0.9 

54 

0.0 

01 

0.06 

6 

0.00 

6 
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Model 


df 

A dj RMSi ARMS CFI 

AC/7 

SR 

MR 

A SRA Mo 

d. 

EP 

C 

Po 

we 

Pat 

h 












Ind 


r 




9 













Pscalarl 

827. 

3 

7 

165 

22 

0.0 

0.011 

0.9 

-0.0 

0.07 

0.01 

61. 

-0. 

0.2 

FC1 

09* 

I 

i 

.14* 

63 

28 

26+ 

8 

2+ 

39 

58 

7 

:GY 

Pscalar2 

744. 

3 

6 

9 

82. 

20 

0.0 

0.004 

0.9 

-0.0 

0.07 

0.00 

24. 

0.2 

0.4 

FC3 

60* 

65* 

56 

43 

11 + 

2 

6 

85 

8 

2 

:GY 

Pscalar3 

708. 

07* 

3 

6 

7 

46. 

12* 

18 

0.0 

54 

0.002 

0.9 

49 

-0.0 

05 

0.06 

9 

0.00 

3 





Groups: Guyana, Jamaica and Trinidad & Tobago 









configural 

628. 

35* 

3 

2 

7 



0.0 

52 


0.9 

57 


0.05 

9 






Metric 

666. 

3 

C 

38. 

24 

0.0 

-0.00 

0.9 

-0.0 

0.06 

0.00 





41* 

O 

i 

05 

51 

1 

56 

01 

7 

8 





Scalar 

769. 

3 

7 

102 

24 

0.0 

0.007 

0.9 

-0.0 

0.07 

0.00 

51. 

-0. 

0.2 

FC1 

14* 

I 

5 

.73* 

58 

39 

17+ 

4 

7 

42 

51 

9 

:GY 

Pscalarl 

722. 

3 

7 

55. 

22 

0.0 

0.002 

0.9 

-0.0 

0.07 

0.00 

11. 

0.1 

0.4 

FC3 

30* 

3 

89* 

53 

50 

06 

0 

3 

94 

9 

3 

:GY 

Pscalar2 

708. 

3 

7 

41. 

20 

0.0 

0.001 

0.9 

-0.0 

0.06 

0.00 

12. 

-0. 

0.9 

BI3: 

10* 

i 

i 

69 

52 

52 

04 

8 

1 

75 

11 

1 

TT 

Groups: Guyana, Jamaica and Barbados 










Configural 

631. 

81* 

3 

2 

7 



0.0 

53 


0.9 

55 


0.06 

1 






Metric 

674. 

3 

C 

42. 

24 

0.0 

-0.00 

0.9 

-0.0 

0.07 

0.01 





43* 

O 

i 

62 

52 

1 

53 

02 

1 

0 





Scalar 

844. 

3 

7 

170 

24 

0.0 

0.011 

0.9 

-0.0 

0.08 

0.01 

65. 

-0. 

0.2 

FC1 

98* 

l 

5 

.56* 

63 

27 

26+ 

1 

0+ 

07 

60 

7 

:GY 

Pscalarl 

760. 

3 

7 

86. 

22 

0.0 

0.005 

0.9 

-0.0 

0.07 

0.00 

24. 

0.2 

0.4 

FC3 

73* 

l 

3 

30* 

57 

42 

11 + 

8 

7 

11 

6 

6 

:GY 

Pscalar2 

723. 

3 

7 

49. 

20 

0.0 

0.002 

0.9 

-0.0 

0.07 

0.00 

14. 

-0. 

0.9 

BI3: 

64* 

i 

i 

21* 

54 

48 

05 

4 

3 

31 

11 

5 

BB 

Groups: Jamaica, Barbados and Trinidad & ' 

Tobago 









Configural 

619. 

35* 

3 

2 

7 



0.0 

52 


0.9 

59 


0.05 

9 






Metric 

652. 

36* 

3 

5 

33. 

01 

24 

0.0 

51 

-0.00 

1 

0.9 

59 

0.0 

00 

0.06 

6 

0.00 

7 
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Model 


df 

A dj RMSi ARMS CFI 

AC/7 

SR 

MR 

A SRA Mo 

d. 

EP 

C 

Po 

we 

Pat 

h 












Ind 


r 


Scalar 

721. 

1 

3 

7 

69. 

24 

0.0 

0.003 

0.9 

-0.0 

0.07 

0.00 

12. 

-0. 

0.6 

FC3 

73* 

I 

5 

37* 

54 

51 

08 

1 

5 

31 

16 

0 

:BB 

pscalarl 

707. 

52* 

3 

1 

3 

55. 

16* 

22 

0.0 

53 

0.002 

0.9 

53 

-0.0 

06 

0.07 

0 

0.00 

4 

11. 

15 

0.1 

3 

0.7 

4 

BI3: 

JA 

Pscalar2 

694. 

3 

7 

42. 

20 

0.0 

0.001 

0.9 

-0.0 

0.06 

0.00 

13. 

0.1 

0.8 

PEI 

64* 

i 

28* 

52 

55 

04 

9 

3 

75 

3 

4 

:BB 






Four-Group 

Model 








Groups: Jamaica, 

Guyana, 

Barbados and Trinidad & Tobago 







Configural 

839. 

47* 

4 

3 

6 



0.0 

53 


0.9 

56 


0.06 

0 






Metric 

896. 

4 

7 

56. 

36 

0.0 

-0.00 

0.9 

-0.0 

0.06 

0.00 

9.1 

-0. 

0.3 

EE2 

33* 

1 

2 

85 

52 

1 

55 

01 

9 

9 

9 

19 

6 

:BB 

Pmetricl 

881. 

4 

C 

41. 

33 

0.0 

-0.00 

0.9 

0.0 

0.06 

0.00 





11* 

o 

9 

63 

51 

2 

56 

00 

7 

7 





Pscalarl 

1069 

5 

0 

2 

188 

30 

0.0 

0.009 

0.9 

-0.0 

0.07 

0.00 

62. 

-0. 

0.2 

FC1 

.71* 

.61 

60 

36 

20+ 

6 

9 

97 

61 

6 

:GY 

Pscalar2 

983. 

4 

Q 

102 

30 

0.0 

0.004 

0.9 

-0.0 

0.07 

0.00 

22. 

0.2 

0.4 

FC3 

85* 

O 

9 

.74 

55 

47 

09 

2 

5 

00 

7 

1 

:GY 

Pscalarl 

946. 

4 

Q 

65. 

27 

0.0 

0.001 

0.9 

-0.0 

0.06 

0.00 

13. 

0.1 

0.7 

PEI 

66* 

6 

55 

52 

51 

05 

9 

2 

64 

3 

9 

:BB 


* significant at the 5% level. Mod. Ind. - modification index. EPC - Expected parameter change. + 
significant change in fit index. GY - Guyana. BB - Barbados. JA - Jamaica. TT - Trinidad & 
Tobago. Pmetric- Partial metric invariance. Pscalar- Partial scalar invariance. The chi-square 

tests for the nested model - A^ 2 - are done with a Bonferroni correction for multiple tests. 
Therefore p-value = rr for an overall 5% test. 


'A df 


As in the case of metric invariance, full scalar invariance of the measurements is observed for the 
comparisons of Jamaica with each of Trinidad & Tobago and Barbados and for the comparison of 
Barbados with Trinidad & Tobago. Both the item loadings and the intercepts are therefore equal 
when these groups are compared in a pair-wise manner. Violations of scalar invariance occur 
whenever Guyana is paired with another territory. These violations are due generally to the 
intercepts of the first and third indicators of FC (FC1: In general, my University campus has 
support for mobile learning. FC3: / have the knowledge necessary to use m-Learning.) (Table 7). 
FC1 has a lower mean in Guyana than the other groups whereas the mean of FC3 is higher in 
Guyana compared to Barbados and Trinidad & Tobago. 
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An interesting result obtained from the three-group analyses is that the measurements are not fully 
invariant when Jamaica, Barbados and Trinidad & Tobago are estimated simultaneously. This 
result is interesting since there is full scalar invariance when any pair-wise combination of these 
groups is considered. Pair-wise Ml therefore does not guarantee, Ml if the elements of the pairs 
are estimated together. Full metric invariance of the measurements is demonstrated in the 
simultaneous estimation of these groups, but not full scalar invariance. Scalar invariance is 
hindered by the intercepts of the third indicator of FC (FC3) and the third indicator of Bl (BI3: / 
have a plan to use m-Learning technologies in the near future.). The reported average level of 
knowledge about mobile learning technologies (FC3) is lower in Barbados whereas the students 
from Jamaica report a higher likelihood of having a plan to use mobile learning in the near future. 

Whenever, Guyana is included as one of the three groups, the same Ml violations encountered in 
the two-group comparisons are encountered. In each, case, full scalar invariance is hindered by 
the intercepts of FC1 and FC3. However, full metric invariance is obtained in the three-group 
models which include Guyana as long as both Barbados and Trinidad & Tobago are not the 
remaining two groups. When the models for these two groups are estimated simultaneously with 
Guyana, only partial metric invariance is achieved, due to non-invariance of the loading of the 
second indicator of EE in the Barbados group. However, if the models for Guyana, Jamaica and 
Barbados are estimated simultaneously, full metric invariance is achieved. 

In some of the three-group models, BI3 and PEI are flagged because they have large modification 
indices when scalar invariance is evaluated (see Table 7). However, apart from the case already 
discussed in which the item is indeed non-invariant, we do not regard the respective item 
intercepts as non-invariant in these remaining cases because the power is above 80% while the 
expected parameter change is not large. 

The final model estimated contains the four groups under study. The results (Table 7) are largely 
consistent with what is already discussed, but some of the details are lost. Full Ml is not obtained 
from the simultaneous estimation of the models for the four groups. Partial scalar invariance is 
achieved due to the loading of the second indicator of EE. This loading is lower in the Barbados 
group. The model also requires two additional modifications when scalar invariance is evaluated. 
These modifications are made to the intercepts of the first and third indicators of FC. The intercept 
of FC1 is lower whereas the intercept of FC3 is higher in Guyana. 

Comparisons of the Means of the UTAUT Factors 

In addition to evaluating the validity and comparability of the UTAUT factors in the Caribbean 
region, we provide comparisons of the average levels of the factors between the groups. The 
comparisons of the factor means are based on the results of the four-group model. Given that only 
partial scalar invariance is established, only the invariant items contribute the factor means (see 
Table 8). 

As observed in Table 8, there are several significant differences in the average levels of the 
UTAUT factors between the territories, but there is also a notable lack of significant difference in 
many cases. PE is highest in Guyana followed by Jamaica. The students from Guyana in 
comparison to those from Barbados and Trinidad & Tobago in particular appear to feel more 
strongly about the usefulness of mobile technology in education. EE is also higher in the Guyana 
than both Barbados and Trinidad & Tobago. While the average level of this factor is similar 
between Guyana and Jamaica and between Jamaica and Barbados, the students from Jamaica 
feel more confident in their ability to use mobile technologies than the students from Trinidad & 
Tobago. There are no differences in the levels of the social factors (SF) except when Guyana is 
compared to Barbados and Trinidad & Tobago. In both cases, the students from Guyana report 
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higher levels of social support. The only difference in the evaluations of the facilitating conditions 
occurs for the comparison of Guyana with Barbados. In this instance, the students from Guyana 
are less optimistic about the conditions in support of mobile learning. Finally, Bl is higher in 
Guyana than both Jamaica and Trinidad & Tobago, but higher in Barbados than Jamaica. 

Table 8: Comparisons of the Factor Means 

Factor 



Group 





Jamaica < 

Guyana 

Barbados 


Trinidad & Tobago 

Mean 

Mean 

diff. 

SE 

Mean 
t diff. 

SE 

t 

Mean 

diff. 

SE 

t 

Baseline: Jamaica 









PE ref 

0.08* 

0.05 

1.80 -0.17** 

0.05 

-3.64 

-0.16** 

0.05 

-3.21 

EE ref 

0.06 

0.07 

0.88 -0.11 

0.06 

-1.63 

-0.14* 

0.06 

2.23 

SF ref 

0.09 

0.09 

1.06 -0.08 

0.08 

-0.92 

-0.06 

0.08 

-0.75 

FC ref 

-0.04 

0.10 

-0.45 0.13 

0.09 

1.42 

0.09 

0.10 

0.92 


0.26* 








Bl ref 

* 

0.09 

2.96 0.16* 

0.09 

1.72 

0.12 

0.09 

1.25 

Baseline: Guyana 









PE 

ref 


-0.26** 

0.05 

-5.49 

-0.24** 

0.05 

-4.96 

EE 

ref 


-0.16** 

0.06 

-2.57 

-0.20** 

0.06 

-3.17 

SF 

ref 


-0.17** 

0.08 

-2.06 

-0.15* 

0.08 

-1.84 

FC 

ref 


0.17* 

0.09 

1.92 

0.14 

0.10 

1.39 

Bl 

ref 


-0.10 

0.09 

-1.20 

-0.15** 

0.09 

-1.67 

Baseline: Barbados 








PE 



ref 



0.04 

0.10 

0.38 

EE 



ref 



-0.06 

0.10 

-0.61 

SF 



ref 



0.01 

0.09 

0.15 

FC 



ref 



-0.05 

0.11 

-0.44 

Bl 



ref 



-0.04 

0.09 

-0.46 


**Significant at the 5% level. ‘Significant at the 10% level. SE - Standard error, t - test statistic 
from the t-distribution. ref - indicates the reference group for the comparisons. The mean of the 
reference group is set to 0. 


In the UTAUT model, Bl impacts directly on technology adoption. Given the results of the 
comparisons of the means, the students from Guyana seem poised to adopt mobile learning. The 
sizes of the coefficients (0.24 to 0.26) when Guyana is compared to the other territories are also 
the largest of all the mean differences. Given the sizes of the t-statistics corresponding to these 
differences (Table 8), the effect sizes are also expected to be large. 

Aside from the differences in Bl, a few notable patterns emerge for the comparisons of the factor 
means. Firstly, Barbados and Trinidad & Tobago form homogeneous groups with respect to the 
means of all the factors. Secondly, Jamaica is distinguished from the other groups with respect to 
two factors in each case. Significant differences from Jamaica occur for PE in all cases and with 
respect to Bl in comparison to both Guyana and Barbados. The only remaining significant 
difference involving Jamaica is in relation to EE with Trinidad & Tobago. Although it retains a few 
distinguishing characteristics, Jamaica appears to have many similarities with each of the other 
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territories. In contrast, the results for Guyana contains several distinguishing points. The mean 
levels of the factors in Guyana differ from those in Barbados and Trinidad & Tobago in relation to 
all but one factor each. However, Guyana and Jamaica have similar means for most of the UTAUT 
constructs. 


DISCUSSION 

With the exception of the facilitating conditions factor which has relatively low convergent validity, 
the UTAUT constructs are recovered adequately from the data in each territory (Thomas et al. 
2013). Apart from the measurement validity issues of the facilitating conditions, this finding is 
similar to that of Oshlyansky et al. (2007) who focus on a combination of Western and 
non-Western countries. The UTUAUT measures are therefore generally both valid and reliable 
and this supports the generalizability of the measurements to the Caribbean region. The UTAUT 
theory may therefore be used to evaluate mobile learning adoption in the region. However, there 
are some caveats to this general conclusion. 

The consistently low convergent validity of the facilitating conditions factor is enough to raise 
concerns about the measurement of the factor. While there is a need for improvement in the 
measurement of FC, discarding it altogether is not recommended (Thomas et al. 2013) especially 
since the construct provides unique information. Closer inspection of the items reveals that they 
focus on a combination of sources of support that are external (FC1 and FC4) and internal to the 
individual (FC2 and FC3). Splitting this construct into two separate factors capturing the external 
and internal facilitating conditions respectively may result in more appropriate measurement of the 
facilitating conditions in the Caribbean region. This approach requires a modification of the UTAUT 
in relation to the measurement of the facilitating conditions and it should be explored in future 
studies. However, researchers should also attempt to identify two additional items; one to go along 
with FC1 and FC4 to measure external conditions and one to go along with FC2 and FC3 to 
measure internal conditions. This is suggested because the resulting two factors will be 
under-identified if only two items are used with confirmatory factor analysis. In some cases, further 
restrictions (loadings equal 1) will need to be imposed for the model to be estimated successfully if 
only two items are used. 

It is also noteworthy that the expected parameter change for the first indicator of the facilitating 
conditions (FC1: In general, my University campus has support for mobile learning.) when scalar 
invariance is attempted is large. This suggests that in comparison to the other groups, the 
Guyanese students are much less optimistic about the level of support for mobile learning at their 
campuses. This is supported by the comparisons of the factor means. The facilitating conditions is 
the only construct on which the Guyanese students report a lower average than the students of 
any other territory. This particular observation may be reflective of the country’s comparatively 
poor standing on the ICT development index and rank together with only approximately 34% of the 
population using the Internet (International Telecommunication Union 2013; International 
Telecommunication Union 2012). 

A final caveat to the generalizability of the UTAUT measures, is the consistently low loading of the 
third indicator of social factors (SF3: University teachers are supportive of the use of mobile 
technologies). This indicator captures the influence of university teachers on the adoption of 
mobile learning. The supportiveness of university teachers therefore appears to play a limited role 
in determining the social factors responsible for mobile learning adoption in the region. This item 
was modified to fit the university context, but it is apparent that a replacement may be necessary 
for mobile learning studies in the Caribbean. In the Caribbean, mobile learning has not yet become 
integral to higher education and the general poor performance of the third social factors item may 
be due to domain specificity. It may be that the teachers are themselves less technology savvy 
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and that the students do not look to them for inspiration in this regard. The low loading of this item 
in Guyana was highlighted previously (Thomas et al. 2013) and the consistency of this finding 
across the territories in this study, suggests that there is a need to modification of the item to 
improve its validity. Such modification is necessary at least for mobile learning studies, but we 
advise against modifying this item for the general application of the UTAUT model until it is tested 
more widely in the region in other domains. 

The UTAUT measures are generally comparable across the territories; however, one violation of 
metric invariance is encountered. This is a limitation on the robustness of the measures (Byrne & 
Watkins 2003; Van de Vijver & Tanzer 2004). This violation of metric invariance occurs for an item 
that has a quite high standardised loading (greater than 0.80). We do not believe that this 
particular parameter will bias research results substantially. In spite of this, researchers should still 
be cautious about pooling data across the territories (Byrne et al. 1989). Under partial metric 
invariance, cross-national comparisons of structural relationships is permitted. Researchers can 
therefore compare the regression effects included in the UTAUT model among the territories. This 
is an important result since these effects are most often the subject of research. Based on the 
results, comparisons of structural relationships under full metric invariance can be done with any 
group of three of the territories except when Barbados and Trinidad & Tobago are included with 
Guyana. 

Comparisons of the mean levels of the factors are also important. Such comparisons can indicate 
the relative standing of the territories on the factors so that areas of focus for mobile learning 
intervention may be identified. Given that only partial scalar invariance is found, comparisons of 
the mean levels of the factors are likely to be biased unless adjustments are made (Vandenberg & 
Lance 2000). In particular, only pairs of territories excluding Guyana may be compared under full 
scalar invariance. Once Guyana or more than two territories are included adjustments for scalar 
non-invariance are necessary (Byrne et al. 1989). The construct that is most affected is the 
facilitating conditions primarily due to the Guyana group. This construct may also show 
non-invariance in Barbados as observed in the three-group models. In addition, behavioural 
intention may also show non-invariance in Jamaica when Guyana is excluded. The reason for this 
is that the violations are more severe in Guyana. This is notable given that Guyana performs the 
poorest of the four territories on both ICT rankings and ICT development (Table 1). The difference 
in the ICT environment appears to indeed affect usage and user experience and thus accounts for 
the results obtained. The issue of usage and user experience across the Caribbean territories 
should be investigated in the future 

Once partial invariance is invoked, the freed items no longer contribute to the substantive 
comparisons. This is true for the results provided on the mean differences between the UTAUT 
factors. The facilitating conditions is most affected and as such the results should be interpreted 
with care. It is also important to note that methods such as analysis of variance or the use of sum 
scores or averages for the UTAUT construct, ignore the measurement issues highlighted in this 
paper. These approaches are therefore not optimal for comparing the regression effects in the 
UTAUT model or the mean levels of the measures between the Caribbean territories. Methods 
that permit adjustments for lack of measurement invariance should be employed. 


CONCLUSION 

With the exception of the facilitating conditions, the UTAUT measures exhibit adequate reliability 
and the factors are adequately recovered in each territory. The interpretations of the items per 
factor are generally similar across the groups, but similarity of the item intercepts is more 
problematic. With potentially one adjustment, the structural relationships among the UTAUT 
factors may be compared, but more adjustments are required for comparisons of the factor 
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means. Pair-wise pooling of the data across the groups is justified except for some cases in when 
Guyana is involved. However, once more than two groups are included, researchers should be 
cautious about pooling the data especially when the factor means are to being studied. We 
conclude overall that the UTAUT model may be used in the Caribbean region but that researchers 
should focus attention on improving the measurements. In particular, the UTAUT model may be 
modified for the Caribbean region to allow the facilitating conditions to be measured by two 
separate factors; one capturing the contribution of the individual and the other capturing the 
contribution of the environment. This represents a departure from the UTAUT theory as it relates 
to the measurement of facilitating conditions in the Caribbean context. However, the need for this 
modification should be investigated in domains other than mobile learning before it could be 
generalised. 


LIMITATIONS 

This study has three important limitations. Firstly, university campuses that come under the UWI 
brand entirely compose the samples of two territories. We do not anticipate this this has affected 
the results substantially (see description of data) but the possibility is not altogether eliminated. 
Secondly, response styles are not controlled (Van Vaerenbergh & Thomas 2013). Response 
styles are examples of method bias which can affect the extent of Ml obtained and group 
comparisons regardless of whether or not Ml is demonstrated (Thomas et al. 2014). However, 
corrections for response styles have not become commonplace in research and data collection 
instruments do not generally cater for this. Thirdly, we used student samples which is necessary 
for the topic investigated. As such, the results may not be generalised to other groups and to other 
domains. Furthermore, the issue of self-selection in web surveys may further affect the results. 
Researchers should examine both the impact of response styles on the cross-national 
comparability of the measures and the generalizability of the findings in domains other than mobile 
learning and such studies should include other Caribbean territories and other university 
campuses. 
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